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Abstract 

This paper examines issues about leadership preparation by focusing on existing research and 
unaddressed questions about what happens both inside and outside cohorts, a model for program 
delivery used in many universities today. This paper seeks to stimulate dialogue about the impact 
of cohort participation on learning outcomes, professorial roles, and practice. The objective is 
two-fold: (a) to design a nationwide research agenda about the influence of cohort delivery models 
on the professional practice of school leaders and (b) to develop a knowledge base about effective 
strategies for designing and implementing the professional development programs for school 
leadership in the 2 1st century. 
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Performance Outcomes and Accreditation: 

A National Research Agenda 

The national spotlight is aimed at educational leadership preparation programs as 
accrediting agencies and state departments of education continue to demand greater 
accountability (National Policy Board for Educational Administration, 1998; Van Meter & 
Murphy, 1997; Usdan, 2002). At the same time, many program faculty are developing varied 
and innovative instructional strategies and organizational structures to prepare school 
administrators to lead schools in these complex times (Jackson & Kelley, 2002; Murphy, 1993; 
Peterson, 2002). Calls for reform and program assessment within university-based leadership 
preparation programs have resulted in moderate changes but few national studies that show value 
added — that programs actually make a difference (Bogotch, 2001; Forsyth & Murphy, 1999; 
Glasman, Cibulka, & Ashby, 2002; McCarthy, 2001). 

In the face of these pressures and challenges, program faculty have begun to focus on the 
connection between preparation activities and actual leadership practices. Yet, faculty doing so 
face a daunting challenge: preparing leaders capable of creating effective learning environments 
for all learners, establishing socially responsible policies and practices, and collaborating with 
multiple stakeholders in their communities. Facing these increasing demands, public 
accountability, and the spotlight of reform cast on the role of school leadership in school 
improvement, school systems are finding that fewer and fewer qualified educators want to accept 
these important leadership roles (ERS, NAESP, & NASSP, 2000; Fenwick, 2000; Grogan & 
Andrews, 2002; Winter, Rinehart, & Munoz, 2001). In addition, challenges to the efficacy of 
university-based administrator preparation programs (Brent, 1998; Haller, Brent, & McNamara, 
1997) cannot be countered without empirical evidence that preparation for leadership leads to 
effective school leadership (Muth & Barnett, 2001; Young, Peterson, & Short, 2002). 

Clearly, no one best model exists for expanding the pool of qualified and willing 
candidates and preparing them for the challenges that they will face. However, one of the 
increasingly popular recruitment and instructional delivery strategies is the use of cohorts. 
Estimates suggest that over 50% of graduate educational leadership preparation programs in the 
United States are using this approach (Barnett, Basom, Yerkes, & Norris, 2000; McCarthy, 
1999b). Despite instances of increased academic competition (Hill, 1995), power struggles 
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(Teitel, 1995), and faculty workload (Burnett, 1999), many benefits are afforded by cohort 
delivery. For example, the cohort structure has the potential to strengthen curriculum integration, 
team teaching, and course scheduling (Martin, Ford, Murphy, Rehm, & Muth, 1997; Yerkes, 
Basom, Norris, & Barnett, 1995). Cohort members’ learning also can be positively affected, 
including their scholarship and reflective abilities (Burnett, 1989; Hill, 1995; Leithwood, Jantzi, 
& Coffin, 1995; Norton, 1995), interpersonal relationships (Horn, 2001a), professional networks 
(Muth & Barnett, 2001), and persistence in program completion (Dorn, Papalewis, & Brown, 
1995; Norton, 1995). 

Despite numerous accounts of the viability of cohorts as an instructional methodology, 
empirical data are largely unavailable to substantiate such claims. Most research has relied on 
small samples and on the perceptions of those using cohort delivery models (Barnett et al., 2000). 
Attacks on the profession declare emphatically that empirical evidence fails to substantiate 
how — or even if — educational leadership preparation programs affect student-learning outcomes 
(short-term effects) and future practice (long-term effects) as school leaders (Black & English, 
1986; Brent, 1998; Haller et al., 1997). The research gap is significant for educational leadership 
programs in general — and cohort programs specifically— in terms of their short- and long-term 
efficacy. 

Here, we examine these and other issues, focusing both on what research tells us so far 
and what additional research is needed for us to increase understanding of the impact of cohorts 
on learning outcomes, professorial roles, and practice. We examine existing research and 
unaddressed questions about what happens both inside and outside the cohort experience. The 
primary point, though, is that we — faculty in educational administration — need to do more 
research on our primary practice: preparing people for administrative leadership positions in our 
nation’s schools. Research on cohorts demonstrates one way to undertake this task. 

In addition, as new practices such as cohorts emerge, clear research agendas will help us 
build evidence about the utility of these practices. The broader these research agendas, the more 
likely we are to generate a solid knowledge base. And this knowledge base will permit us to 
choose the most effective strategies for designing and implementing the professional 
development programs needed for schools in the 2P‘ century. 
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A Definitional Note 

Before continuing, an important distinction needs to be made about the use of the term 
cohort here. Although some leadership preparations programs use the word cohort to describe a 
group of students selected from particular local educational agencies or students who enroll in a 
program during a specific semester, these students generally do not begin and complete their 
programs together as an intact community of learners. As they progress through their preparation 
programs, they generally interact with different students in various classes. Such groups of 
students are usually excluded from research on cohorts because their learning environments are 
transient. That is, over the course of a program of studies, their classrooms vary considerably — 
by subject, professor, and contact with the field — as do the students who populate them. 

Thus, this paper reports research only about “closed cohorts” (Norris & Barnett, 1994) in 
which students remain together throughout their preparation program as a single, identifiable 
group. The issue of quasi-cohorts (e.g., student-grouping models that are used during program 
orientation, that track school-district affiliation, or that are created by careful sequencing of 
separate course scheduling) is addressed in later sections of this paper. 

Role of Cohorts in Improving Outcomes 

Although research on what occurs in cohort programs is somewhat sparse, a few 
empirical investigations have examined their use and effects (Barnett et al., 2000; Hebert & 
Reynolds, 1998; Reynolds & Hebert, 1995; Witte & James, 1998). These studies have generally 
found that outcomes are positive; Cohorts have an impact on learning and performance. Much of 
this research has surfaced in the last decade and appears to be located primarily in education as 
cohorts have become a popular way to prepare educators for various roles (Milstein & Krueger, 
1997; Murphy, 1993). What is not “known” or evident from research about participant learning 
in cohort models — or, indeed, in any other models — is whether such learning transfers to 
administrative practice (Bridges & Hallinger, 1992; Hebert et al., 1998; Horn, 2001a; Pounder, 
1994; Twale & Kochan, 2000). 

A premise for using cohorts in administrator-preparation programs is that keeping 
students together as a unique group of about 15 to 25 learners enhances professional learning and 
skill development (Barnett et al., 2000; Kelley & Peterson, 2000; Norris & Barnett, 1994; Peel, 
Wallace, Buckner, Wrenn, & Evans, 1998). Investigations of preparation programs delivered as 
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coherent, sequenced curricula for unique cohort groups suggest increased student completion 
rates (Reynolds & Hebert, 1995) and enhanced learning achievements (Hebert & Reynolds, 

1998) when compared to programs delivered as a series of traditional, separate courses taken by 
individual students whenever desired or available. Another assertion is that the cohort structure 
provides excellent opportunities for aspiring school leaders to learn and practice skills in group 
goal setting, community building, conflict resolution, and culture management (Geltner, 1997). 
Thus, the use of cohorts often is recommended highly in preparation programs in educational 
leadership (Milstein & Krueger, 1993; Murphy, 1993). 

Role of Cohorts in Learning, Preparation, and Practice 

The primary reasons given by faculty for using cohorts are (a) the opportunity for students 
to form strong interpersonal relationships with peers and faculty and (b) the efficiency of 
program delivery (Barnett et al., 2000). Anecdotal evidence, usually provided by students and 
faculty at the close of programs, suggests that the long-term association of learners in a cohort 
creates caring learning climates that support students’ competence and sense of well-being (Crow 
& Glascock, 1995; Norris & Barnett, 1994; Peel et al., 1998). Students in cohorts that function 
well report greater feelings of inclusiveness, more opportunities for collaboration and 
professional networking, and stronger academic performance than they experienced in their 
previous higher educational experiences (Basom, Yerkes, Norris, & Barnett, 1995; Yerkes, 
Basom, Barnett, & Norris, 1995). Studies of group dynamics, group affiliation, participant 
interaction, and personal relationships within cohorts (Basom et al., 1995; Norris & Barnett, 

1 994) also show that the culture of cohorts that function well increases the level of learning for 
all participants. 

Another finding is that the traditional delimitations of instructor and student roles 
routinely dissolve within cohorts while frequently expanding to include school practitioners who 
participate in elements of program development and delivery (Cordiero, Boutiler, Panicek, & 
Salamone-Consoli, 1993). Over time, cohort participants assume greater responsibility for their 
learning — both as individuals and as members of a group— and regularly make known their 
demands for changes in instructional delivery or learning assessments (Barnett et al., 2000). 

Thus, cohort faculty often assume roles as facilitators, mentors, and occasionally mediators 
during cohort meetings that resemble professional development seminars and workshops rather 
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than traditional higher education lecture classes. However, use of the cohort instructional 
delivery model occasionally affect faculty teaching adversely because students are more likely to 
challenge instructors’ authority (Barnett et al., 2000; Barnett & Muse, 1993) due to the strong 
social bonding of cohort groups (Norton, 1995). 

Team learning within a cohort does not emerge simply by grouping students together 
(Basom et al., 1995). Research suggests that learning in a cohort is enhanced when 
environmental influences are addressed and learner-centered instructional strategies are 
implemented (Barnett et al., 2000; Hannafm & Land, 1997; Muth et al., 2001). Thus, cohort 
faculty need to create inviting, risk-safe conditions and address stages of cohort transformation, 
marked by cycles of conflict and cohesion, so that transformative learning can occur (Geltner, 
1994; Lumsden, 1992; Maher, 2001). Linking cohort learning activities to professional practice 
is strengthened through clinical practice experiences, monitored internships, mentoring, and 
networking opportunities (Browne-Ferrigno & Muth, 2001; Lumsden, 1992; Milstein & Krueger, 
1997). 

The cohort model also supports the development of university-district partnerships, often 
linking classroom learning to professional practice, making it easier to structure and manage. 

The changed roles of aspiring school leaders, leadership professors, and educational practitioners 
within collaboratively created cohort programs has the potential of reframing educational 
administration preparation through closer linkage of preservice learning to professional practice. 
Outcomes Assessments: Personal, Programmatic, National 

Cohort programs are apt to incorporate a variety of instructional strategies that include 
integrated course content, team teaching, problem-based learning, reflective strategies, and case 
studies (Hill, 1995; Martin et al., 1997; Yerkes et al., 1995). Cohort programs also appear to 
follow some of the current trends in assessment by incorporating diagnostic inventories and 
individual learning plans (Yerkes et al., 1995). However, a challenge in designing an effective 
cohort program based upon learning through experiences is the selection of the kind of “present 
experiences that live fruitfully and creatively in subsequent experiences” (Dewey, 1938, p. 27). 

By developing expectations about problem-based learning activities linked to real 
problems of practice in K-12 schools (Muth, 2000), faculty enable students to begin early to 
apply theories and to develop skills needed in their future roles as school leaders (Pounder, 
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1994). By integrating group action-research projects (Geltner, 1993; Stringer, 1996) and 
appreciative inquiry activities (Srivastva, Cooperrider, & Associates, 1990) into the curriculum, 
students learn the power of collaborative inquiry (Churchill, 1996) and the importance of careful 
use of data and reflection (Nadler, 1977; Schon, 1991). By working together in various small- 
group and whole-class settings, students learn the challenges of group dynamics when 
membership changes (Muth et al., 2001). Further, the intact learning environment of closed 
cohorts supports long-term developmental activities such as rote playing, simulation, and in- 
basket and point-counterpoint activities (Cordiero et al., 1993) that are more difficult to integrate 
into individual courses where student learning groups are reconstituted each semester. 

To accommodate changes in instructional strategies within cohorts, student learning 
assessments typically include individual and small-group projects and presentations, reflective 
journals, and field-based inquiries and school projects usually presented through portfolios 
(Milstein et al., 1993). Real-world, authentic assessments require students to develop 
collaborative products by sharing papers and resources, gaining peer clarification, working 
through conflict, and seeking agreement through consensus. Authentic assessments help attune 
learning to professional practice (Bransford & Schwartz, 1999). 

Using the cohort model effectively requires program coherence (Dick, 1990; Muth, 

2002b) and faculty engagement in identifying and implementing critical elements that generate 
optimum learning experiences for faculty and students (Barnett et al., 2000; Kelley & Peterson, 
2000). Faculty need to reframe their notion about content covered within individual courses to 
content delivered through appropriate scope and sequence. Collaborative program development 
means that the “onus of responsibility is upon all faculty to contribute and explain why their 
content and ideas need to be included” (Cordiero et al., 1993, p. 27). Thus, determination of 
desired cohort outcomes requires deliberation during program development to ensure that cohort 
activities are aligned with what is to be learned, how it is to be learned, what process is to used, 
and what teachers and students should do (Muth, 2000). Both formative and summative 
evaluations of programs are necessary to ensure that goals are met. 

Another programmatic issue related to cohorts is the increase in faculty workload (Barnett 
et al., 2000; Muth & Barnett, 2001 ; Norton, 1995). Most often, the additional work stems from 
needed coordination among professors, increased student advisement, use of developmental 
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assignments that lead to multiple readings of papers and field reports, and travel time for faculty 
to and from off-campus cohort sites. Faculty perceive workload disadvantages despite cited 
benefits of using the cohort model: predictable course scheduling, easy student enrollment and 
greater retention, as well as opportunities for creativity and flexibility in instructional and 
assessment strategies (Barnett et al., 2000). 

Cohorts generally move through predictable stages of group development and establish 
unique personalities (Maher, 2001; Misanchuk, Anderson, Craner, Eddy, & Smith, 2000; 

Wesson, Holman, Holman, & Cox, 1996). This process may require occasional interventions 
which add to faculty workload. Group dynamics resulting in collusion and cliquishness, 
exclusionary group behavior and norms, breakdowns in communication, conflicting problem- 
solving and work styles, and assumed or assigned roles can impede learning (Browne-Ferrigno & 
Muth, 2001; Scribner & Donaldson, 2001; Teitel, 1995). Additionally, multiple in-progress 
assessments may be required to measure group development and elicit candid discussions about 
what is happening within a cohort (Tipping, Freeman, & Rachlis, 1995) and what needs 
changing. Additionally, initial and continuing group-development activities within cohorts may 
be needed to enhance group processing, cohesion, and learning. These in turn diminish time 
spent on content-oriented learning activities (Barnett et al., 2000), assuming, of course, that 
specific content is more important than learning to learn (Muth, 2000; Smith & Associates, 1990; 
Tosteson, Adelstein, & Carver, 1994). 

Accountability: Personal, Programmatic, National 

Perhaps the most attention has been focused on the effects of the cohort experience on 
students. Claims have been made that academic performance improves (Hill, 1995; Murphy, 
1993; Norton, 1995), important leadership skills are acquired (Barnett et al., 2000), and rates of 
program completion increase (Burnett, 1989; Hill, 1995; Norton, 1995). Although many faculty 
contend that interpersonal relationships flourish in cohorts (Barnett et al., 2000), conflicting 
reports have surfaced. On the one hand, cohorts afford students the opportunity to become a 
cohesive community of learners who bond to one another and thus reduce their professional 
isolation (Kasten, 1992; Leithwood et al., 1995; Milstein & Krueger, 1993; Norris, Barnett, 
Basom, & Yerkes, 1996; Norton, 1995). On the other hand, interpersonal tension and conflict 
may result from personal traumas (Barnett & Muse, 1993) academic competition (Hill, 1995), 
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domination of the learning environment by a few vocal students (Norris et al., 1996; Norton, 
1995), and groupthink (Barnett et al., 2000). 

Closed cohorts are perceived to provide unique opportunities for adult students to engage 
in “democratic, non-hierarchical” (Geltner, 1994, p. 13) communities of learners where all 
participants are valued and empowered. Geltner suggests that in advance of delivering cohort 
instruction, faculty need to reflect carefully on their own values and beliefs as individuals and on 
their behaviors and practices as educators. Empowering a cohort requires changes in the 
relationships not only among the learners but also between instructors and students through 
dissolution of traditional power relationships (Barnett et al., 2000; Geltner, 1994; Muth & 

Barnett, 2001) and between university peers who must engage in collaborative syllabus building. 
Instructors who design educational cohort programs also need to be cognizant of and address the 
special characteristics of adult students (Barnett & Caffarella, 1992; Muth et al., 2001) who need 
acknowledgment of their expertise and active involvement in the learning process. 

Although snapshots exist of some structural characteristics of cohorts, very little is known 
about the developmental nature of the cohort experience or the contextual factors affecting cohort 
stability. In order to gain greater understanding about possible developmental and contextual 
effects, additional research is needed that focuses on the entire cohort experience from faculty 
and program perspectives. For example, as new instructors begin to work with cohorts at various 
points throughout the program, what facilitates or hampers their acculturation? How do group 
cohesion and sense of community develop within cohorts over time? What instructional 
processes or critical events foster or impede the development of a cohesiveness? What are the 
long-term effects on the culture of a department using cohorts? Answers to such questions can 
provide greater insights for program improvement and accountability and help professors of 
educational administration determine cost-benefits of cohort use. 

Accreditation, Accountability, and Cohorts 

Although many faculty hold strong views about the benefits or problems of cohort 
instruction, little empirical evidence exists about the long-term effects of the cohort experience 
on aspiring principals’ future professional practice (Goldschmid & Berberat, 1989; Jackson & 
Kelley, 2002; Muth & Barnett, 2001). A critical piece missing in research about administrator 
preparation is the effects of the cohort experience on participants’ workplace behaviors. 
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Suggestions have been made that cohorts have the potential to develop transformational leaders 

(Basom, Yerkes, Norris & Barnett, 1996; Norris et al., 1996). Anecdotal evidence provided by 

some cohort students suggests “a commitment to transfer . . . learning to their future work roles” 

(Norris et al., 1996, p. 159). Leithwood et al.’s (1995) study of the factors affecting later 

workplace performance also indicates a strong relationship between cohort participation and 

assessment of graduates’ leadership capabilities by their co-workers. Unfortunately, a 

comparison group of non-cohort graduates was not used, making it difficult to ascertain the 

relative potency of the cohort experience. 

Through a comprehensive survey study, Barnett et al. (2000) found that faculty 

representing 60% of the 383 universities in Canada and the United States that deliver 

administrative preparation tend to ignore “how the cohort structure can impact the workplace 

practices of their students” (p. 274). Instead, university faculty tend to focus on how students are 

affected by learning within cohorts and differ markedly in their views about the potential for 

cohorts to prepare students for future leadership roles. 

Little direct evidence was provided by respondents regarding how the actual job 
performance of students had been impacted by the cohort experience. Although many of 
their comments centered on students’ enhanced communication skills and problem- 
solving abilities, faculty members did not clearly identify examples of how or whether 
they had observed these practices occurring in the workplace, (p. 274) 

Thus, research is needed to assess how cohort programs differ from traditional programs 
in participant selection, curriculum, instructional processes, exit criteria, and preparation 
outcomes. For example, what leadership knowledge and skills are brought into the learning 
environment by entering participants, and what leadership knowledge and skills are gained by 
cohort members over time? How does this learning differ from that in traditional settings, or is 
such learning more transferable than that of students exiting more traditional programs with more 
traditional teaching-learning processes? What are the effects of these differences, if any, on 
learning, graduates’ decisions to apply for administrative positions, their selection into 
administrative positions, or their performance once in those positions? Even if such comparative 
analyses cannot be achieved, seeking answers to these questions is essential to determining the 
efficacy or value added of administrator preparation under current models. 
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The call to reframe educational administrative practice (Murphy, 1993; 2002) through 
broadly adopted standards for the preparation, practice, and evaluation of school leaders (Council 
of Chief State School Officers, 1996; Van Meter & Murphy, 1997) requires closer links between 
university-based programs and the field of educational leadership practice. Research is needed to 
examine the perspectives of all stakeholders groups involved in the preparation of future school 
leaders (e.g., cohort students, field-based practitioners, university administrators, district-level 
employers, and policy makers). Data derived from carefully designed research projects are 
needed to understand what external factors (e.g., funding, university policies, and school-district 
policies) support or interfere with the creation and maintenance of cohorts. Such questions are 
important to improve the quality of professional preparation and to address political trends, 
acquire support for cohort-based programming, and ensure that the profession of educational 
leadership prospers and matures. 

Need for Empirical Data 

One of the biggest influences of the recent educational reform movement has been the 
focus on standards and accountability. Reforms have dictated that teachers, administrators, and 
university faculty become more proficient at clarifying learning expectations and creating valid 
and reliable assessments to measure these outcomes. In building the argument that university- 
based preparation programs need to increase their capacity to gather and use performance data, 
we next examine some of the factors influencing data-based decision making, especially the need 
for empirical data to improve programs and strengthen the profession of educational leadership. 
The Era of Data-Driven Decision Making 

Over the past decade, K-12 educators have established standards and developed 
performance assessments across grade levels and subjects. Much of the pressure for such reform 
has been generated from external sources, particularly lawmakers and business leaders. Besides 
state-level policy makers, educational reformers have included the National Governors 
Association, the National Conference of State Legislatures, the National Alliance of Business, 
and the Committee for Economic Development (Usdan, 2002). A prominent feature of the 
standards movement emphasizes using data in decision making. On the one hand, classroom 
teachers have been urged to use “data-driven instruction” (Schlecty, 1997) and create 
assessments early in their instructional planning rather than at the conclusion of teaching units 
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(Borko, Flory, & Cumbo, 1993; Hoachlander, Alt, & Beltranena, 2001). On the other hand, 
principals and school district officials are being held accountable for student performance 
outcomes, not only in making school- or district-level decisions, but also in reporting progress to 
parents, local communities, and state departments of education (Clement, 2002; Hoachlander et 
al., 2001; Hoachlander, Levesque, & Mandel, 1998; Stiggins, 2001). As high-stakes testing 
increases, these data are being used to make decisions about employment, funding, and 
accreditation (Glasman et al., 2002). 

Similarly, institutions of higher education in general, and educational leadership 
programs in particular, are being urged to identify learning outcomes for their graduates and 
create outcomes assessment measures (Loacker, 2000; Palomba & Banta, 1999) which should 
relate to effective performance. Even as accrediting agencies begin to require universities to 
specify programmatic outcomes (North Central Association, 1997), increased pressure is placed 
on programs to enhance access for under-represented racial and ethnic groups while ensuring that 
course work is relevant and degrees can be completed in shorter time periods (Association of 
American Colleges and Universities, 2000; National Research Council, 1996). Recently, the 
National Council for Accreditation of Teacher Education (NCATE) has demanded that 
educational leadership programs incorporate standards and performance assessments (National 
Policy Board for Educational Administration, 1998). 

Empirical versus Anecdotal Data and Reform 

These calls for data-driven decision making come at a time when the profession of 
educational leadership is once again being urged to reform (Usdan, 2002; Young et al., 2002). 
Even while the field has struggled to demonstrate that formal preparation programs make a 
difference in the performance of school leaders and their schools — unsuccessfully, according to 
some observers (Brent, 1998; Haller et al., 1997) — claims have been made that an attitude of 
complacency prevails. These claims of complacency suggest that faculty are content with current 
programs and do not see the need to make major changes, especially those dealing with outcomes 
assessment (Glasman et al., 2002; McCarthy, 1999a). However, this lack of urgency may shift as 
older faculty retire and more women and practitioners enter the profession (McCarthy & Kuh, 
1998; Murphy, 1999). 



ERIC 



14 



Performance Outcomes and Accreditation 14 



Despite the dearth of empirical evidence establishing the impact of leadership 
preparation, efforts are emerging to rectify this problem, including suggestions offered in this 
paper. Currently, one of the most visible grass-roots efforts being undertaken by educational 
leadership faculty is a multi-institutional ad hoc committee. Evaluating the Effectiveness of 
Educational Leadership Preparation (EEELP), led by Robert Kottkamp of Hofstra University and 
Margaret Orr of Teachers College, Columbia University. The intent of the group is “to foster 
within and cross-program evaluations, based on how our programs help to improve the quality 
and effectiveness of public education through leadership development” (Orr, 2002). A similar 
idea has been advanced by Muth and Barnett (2001). Presently, subgroups of the EEELP are 
examining (a) student characteristics, (b) program measures, (c) leadership skill development, 
and (d) impact measures. The first subgroup is examining the entry characteristics of students 
while the other three groups are focusing on how the curriculum and delivery mechanisms affect 
students’ learning when they are enrolled in the program as well as the performance of graduates’ 
schools and districts once students assume formal leadership positions. 

At the same time, a second national endeavor, the National Commission for the 
Advancement of Educational Leadership Preparation (NCAELP), has been established (Young & 
Peterson, 2002). This initiative is a collaborative project comprised of educational practitioners, 
community leaders, businesses, professional associations, universities, and governmental 
associations whose purpose is “to improve the practice of educational leadership through high- 
quality preparation and professional development” (p. 131). To date, NCAELP has 
commissioned several papers and critiques dealing with the current and future state of affairs 
regarding educational leadership preparation and professional development. (See the April 2002 
special issue of Educational Administration Quarterly for the commissioned articles.) Aimed at 
collaboratively identifying and resolving problems associated with leadership development, the 
commission intends to address the importance of learning-focused leadership and develop ways 
to assess and improve programs. 

Empirical Data and the Profession 

These two national efforts, although taking different approaches, expect to collect, 
analyze, and compare data about the performance of aspiring and practicing school leaders. 
Analyses will focus, for example, on the use of standards-based approaches to program 
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improvement (Glasman et al., 2002). Yet to enhance its standing as a profession, the field of 
educational leadership must take much more seriously the collection and use of data. Professions 
monitor themselves and in so doing must have valid and reliable data about the performance of 
their membership (Young et al., 2002): 

Although many program faculty now collect data on how students are progressing 
through their programs and whether or not students are meeting the criteria that guide 
their programs, these forms of evaluation do not reveal how well students will perform 
once they are in the field. . . . Until we have a process for determining whether or not 
educational leadership preparation has any of the impacts that we hope for them, it is not 
likely that we will have adequate information to engage in effective program 
development. And without evidence of success, faculty members may not be willing to 
sustain the extra work many program reforms require, (p. 151) 

Currently, much of the empirical data being collected about programs is being shaped by 
accrediting agencies. Because most states require a license or certificate to become a practicing 
administrator, applicants must demonstrate understanding of and competence in the areas 
covered by state standards. As a result, the content, performance expectations, and assessment 
processes in leadership preparation programs have been strongly influenced by these standards 
(McCarthy, 1999a; Young et al., 2002). Moreover, national standards also are driving programs, 
the most salient of which are those developed by the Interstate School Leaders Licensure 
Consortium (Council of Chief State School Officers, 1996) and NCATE (1995). For instance, 
most states have joined ISLLC or have adopted their standards for school leadership (Murphy, 
2001). In addition, the NCATE accreditation process for educational leadership programs is 
pointedly directive about empirical evidence to support measures of student performance 
outcomes. Unfortunately, because university professors tend not to be seen as important sources 
of information (McCarthy, 1999a; Olsen, 2000), their involvement has been marginal in shaping 
standards, performance outcomes, and assessments. While some critics claim that professors of 
educational leadership are passive observers and are politically disinterested (Usdan, 2002), 
perhaps the recent initiatives, especially the professor-initiated EEELP, will lead to performance 
standards and assessments that professors across the country can embrace. 

Developing a National Research Agenda 

Although noteworthy anecdotal data from students and faculty can be found about the 
benefits of using well-designed and well-implemented cohort programs in educational 
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administration, empirical evidence currently is sparse on the long-term effects of cohort 
experiences on the practice of future educational leaders. On the front end, recommendations for 
improving preparation programs include (a) careful screening and selection of cohort 
participants, (b) continual attention to group-development processes, (c) integration of 
experiential activities and reflection into the curriculum, (d) collaboration between faculty and 
practitioners, (e) continuous assessments of student progress measured against clear standards 
and performance rubrics, and (f) modeling of exemplary learner-centered instructional strategies. 
Given the attention to program-specific research already reported above, on the back end much 
remains to be done to determine the efficacy of cohort structures on performance in the field. 

Measuring transference of cohort-based learning to professional practice in school 
leadership may be difficult, and it surely will be labor intensive, costly, and time consuming. 
Nonetheless, accountability for the effectiveness of professional development programs requires 
better data than passing rates on exams, position placements, or anecdotal data from graduates 
and faculty. Short-term and longitudinal studies are needed to trace and examine the transference 
of students’ learning in cohorts to practice settings and to graduates’ professional practices as 
educational leaders. But following career paths of program graduates is difficult without 
continuous upkeep of appropriate databases. Further broad-based longitudinal studies that link 
findings from multiple sites require careful design and diligent attention by consistently involved 
researchers. 

Assuming (a) that difficulties inherent in longitudinal studies can be overcome, (b) that 
groups of leadership-program faculty will participate in such studies to enhance the efforts of 
isolated researchers and provide better data for program revision and justification, and (c) that 
knowing something about transference of cohort learning to professional practice is important, 
then questions need to be raised about what research is necessary and which strategies might be 
most effective in pursuing that research. For example, what data will best tell the field whether 
transference from program learning to field action takes place? How can this transference be 
determined during a program as well as following the completion of a program? What types of 
data sources will be most beneficial for such research? What types of databases will be needed 
to trace graduates’ career paths? Should such database development and use be program 
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specific, or might it be advantageous to create databases for multiple sites, say statewide or 
regionally, to facilitate collection and use of cohort data nationally? 

Some Specific Questions about Learning in Cohorts 

Advocates of cohorts often claim that they provide “better environments for learning,” 
particularly for group learning that takes place over time. Is the continuity across collaborative 
learning experiences and academic terms made possible by cohort structures sufficiently 
analogous to how people learn in field settings that the probability is greater that what is learned 
in a program can be used in practice? Do programs designed to focus on problems of practice 
represent the knowledge and skills that an effective principal must have? Are these problems 
core vehicles for learning and more likely to produce usable and transferable learning than other 
methods of instruction and learning? If so, what are the best ways to confirm this? For example, 
should we design studies that analyze work tasks in school settings in order to examine how 
work teams acquire and use knowledge and then compare these findings to learning in cohorts? 

Another question is whether researchers can identify ways in which cohort instructional 
design affects actual practice. That is, for example, what instructional processes facilitate 
learning about and demonstrating leadership? During the course of a cohort program, do 
students engage in consequential learning activities that require leadership and actually affect 
practice in field settings? Do activities such as leading a curriculum or legal audit produce 
immediate changes in system practices or support later leadership behaviors in diverse job 
circumstances? What are the best ways to capture these outcomes: observations, interviews, or 
some combination of methods? 

Can particular leadership skills be identified that students are expected to acquire during a 
program? Do these skills directly apply to improving school outcomes and transfer to other 
practice settings? In what ways does a given program encourage reflection on one’s leadership 
and its consequences? Further, are such reflective practices continued when no longer required 
by a program? Do program graduates continue to “journal” and think about and reflect regularly 
on their practice and its effects? If so, does reflective practice make any difference in how well a 
program graduate leads a school? What would be reasonable ways to examine whether 
leadership-skill acquisition or reflective practice transfer to on-the-job settings and have any 
effects there? 
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Cohort advocates also claim that cohort experiences help develop professional networks 
that support graduates when they take administrative positions. Are data available that suggest 
that cohorts accomplish this any more effectively than do non-cohort preparation experiences? 
How might comparative analyses be structured? Could comparative “schematics” of networks 
developed by program graduates be examined for use and outcomes? And how might the 
effectiveness of such networks be assessed? Do they actually make a difference in how cohort 
graduates think about their work and perform day to day once they are on the job? 

Other Questions about Impact on Practice 

What are programs, particularly cohort-based programs, doing to assure that students 
have the skills to improve, for example, classroom teaching? How can program designers and 
evaluators know, additionally, if a teacher improves because of a graduate’s intervention? Then, 
how can a program know that the teacher’s improvement positively affects student outcomes? 
Alternatively, does a graduate’s molding of a school’s culture toward high outcome expectations 
actually improve student performance? What measures can be used to determine these results? 
How can programs determine how effectively cohorts determine these results or support 
knowledge and skill acquisition in these areas? 

The questions can go on and on, depicting multiple arenas in which effective principals 
have to apply their knowledge and skills. Can program instructors backtrack to preparation 
programs to see what learning opportunities might have led to or supported the outcomes of 
interest? On the other hand, can programs ask what knowledge, skills, and dispositions “make” 
an effective principal, perhaps using standards developed by various national and state agencies? 
If so, can they test whether cohort or other programs effectively implement learning opportunities 
that connect to these standards? Finally, is it possible to establish whether such standards and 
instructional practices actually have an impact on practice? To demonstrate this impact, 
programs will have to have clear plans at the outset of a particular cohort — or other-program 
that connect instructional intent with later outcomes. 

Data Collection Processes and Methods for a National Research Agenda 

Obtaining meaningful data about student performance in educational leadership 
preparation programs requires a balance between local autonomy and standardization. Using 
only standardized measures, such as paper-and-pencil tests, has been criticized as an ineffective 
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way to determine an individual’s ability to lead a school organization. Reliance on such 
measures also tends to homogenize programs (Horn, 2001b). What is needed is a multi- 
dimensional approach to data collection. Thus, the shared expertise of practitioners and 
professors, informed by research and theories, should drive the data collection process. Outlined 
below are some of the critical issues to consider when obtaining data at the program, regional, 
and national levels. In addition, the importance of longitudinal research designs and suggestions 
for ways of obtaining comparable data across programs are discussed. 

Program-Level Data Collection 

An ideal guiding principle for a coordinated data collection process is to allow individual 
programs to collect data appropriate to their context. Such efforts could be coordinated by state 
accrediting agencies, which establish performance standards and minimum performance 
outcomes, supplying the foundation for collaborative efforts. While many states have adopted 
the ISLLC standards (Murphy, 2001), some unrest still exists among states, professors, and 
professional associations about whether these standards reflect what should be taught and 
assessed in programs (Glasman et al., 2002). Further, individual programs may tend to focus 
preparation on core concepts such as the political dimensions of leadership, change and 
innovation, instructional leadership, or social justice. Hence, at the program level, faculty are 
urged to design assessments that blend data for external audiences and for local program 
improvement (Glasman et al., 2002). 

Program-level data allow faculty to conduct self-evaluations, which can provide feedback 
on program coherence (Muth, 2000) and lead to program redesign. One recent example of the 
method of a standards-based program of self-evaluation has been advanced by Glasman et al. 
(2002). Their program asks students to compile data about a set of core leadership standards, 
which are evaluated by a faculty committee. For each of the core standards — vision, culture, 
organizational management, collaboration, contexts, ethical behavior, and work experience — 
faculty identify the performance expectations emphasized in the program and the criteria used to 
judge students’ performance on these standards. With the guidance of the faculty, students 
develop a leadership portfolio, which provides evidence related to the performance expectations 
for each of the seven standards. The portfolio artifacts include student-produced written 
materials, audio tapes, and videotapes as well as assessments of their field-based experiences by 
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educators who worked with them during their clinical practice. Based on the results of these 
assessments, faculty committees may request more information from those students who have not 
met a standard. In addition, program faculty may alter the curriculum and instructional delivery 
for those standards where many students are struggling. Although this approach is extremely 
ambitious, it provides a working model for programs to consider in building a continuous- 
improvement, standards-based evaluation process. 

Regional Data Collection 

Besides collecting information about student performance that is program specific, 
comparable data can be developed across programs within a geographical region. In many cases, 
these regional standards or performance criteria are those advanced by state licensing agencies. 
While most program faculty view state accreditation as an overly regulatory, low-stakes endeavor 
(Glasman, et al., 2002), faculty might be more motivated to engage in the process if it were seen 
as a means for program renewal and improvement. Because most state-level accrediting agencies 
specify particular standards for graduates of educational leadership preparation, programs within 
a state or region might band together to determine appropriate types of common data needs and 
collect such data across the programs. If designed within the larger, national context, such efforts 
might have even greater potential for impact on practice. 

Through collaborative efforts, faculty in different institutions can learn what others are 
doing, seek funding support together to assist in their data collection efforts, and incorporate a 
“clearinghouse” structure for sharing and developing preparation and assessment practices (Muth 
& Barnett, 2001). Recent funding opportunities from the Wallace-Reader’s Digest Funds (2001), 
the Southern Regional Education Board (SREB) (2001), and other foundations provide models 
for helping faculty with such regional initiatives. The EEELP and NCAELP could foster such 
regional efforts as more localized and sensitive ways to address local and regional program needs 
and outcome preferences. 

National Data Collection 

To build the reputation of the educational leadership profession, a national database needs 
to be developed that facilitates the collection, maintenance, and use of comparable data. These 
data can focus program-improvement efforts as well as publicize the performance of program 
completers. Such efforts, however, undoubtedly will require the collaboration of multiple 
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constituencies, including educational leadership faculty and college administrators, practitioners, 
and professional associations. The biggest challenge in creating a national database is gaining 
the cooperation and participation of significant numbers of institutions. One approach would be 
to have an existing accrediting agency, such as NCATE, take the lead. On the one hand, 
participating colleges and universities already must submit evidence of student performance 
outcomes on a common set of NCATE standards, increasing the possibility of obtaining 
comparable data across programs for use in national assessments of program outcomes and 
impacts. On the other hand, because the data format and types of information may not be 
uniform across programs, comparability of data could be difficult to achieve. 

A different tack might involve a non-accrediting organization maintaining a national 
database. For example, the UCEA Center for the Study of Patterns of Professional Preparation in 
Administration has a record of collecting and reporting data about trends in leadership 
preparation. If the center undertook such data gathering, the scope of its work would expand 
beyond UCEA’s current institutions and capacity, and substantial increases in funding would be 
required. At present, UCEA centers are self-supporting and do not receive much financial 
assistance from the organization or from their host universities. Even so, this more centralized 
alternative might be easier to establish and have wider reach than would multiple separate 
regional efforts. 

In addition, the newly formed NCAELP might be particularly appropriate for 
coordinating such a national effort. Part of NCAELP’ s charge is to develop action plans for 
program improvement and evaluation (Young et al., 2002). The commission is in its infancy 
and, without substantial resources, will not be able to coordinate a massive, nationwide data- 
collection and research-management effort. Yet, either NCAELP or UCEA would have the 
potential for concerted, collective, national action, just what educational administration requires 
to develop the structures and processes necessary to professionalize the field. 

Longitudinal Design 

Besides determining the types of data to obtain, serious attention must be devoted to 
when and how evidence should be collected. For example, it is evident that longitudinal research 
designs are required to assess the value-added nature of preparation alternatives (Orr, 2002). A 
framework for evaluating professional development programs, proposed by Guskey and Sparks 
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(1991), might serve as a template for the types of data to collect at different points in time. In 
particular, Guskey and Sparks suggest that information be collected about how individuals have 
been affected (i.e., impact of the learning environment and its relation to learning outcomes) and 
how their learning has translated into practice (i.e., impact of innovations, application of skills in 
the workplace, influence on student learning, etc.). This framework has direct implications for 
graduate programs because data from program participants will need to be collected at multiple 
points in time: (a) prior to program admission, (b) during the program, (c) at program 
completion, and (d) at the time when graduates actually engage in leadership positions. 

Rarely are baseline data collected about educational leadership preparation program 
candidates that would allow comparisons with data collected at later points in a program, 
following its completion, or after program graduates are in leadership positions. While many 
programs use evidence about applicants’ academic potential (e.g., GRE scores, grade point 
averages, and letters of recommendation) and their leadership capabilities (e.g., recommendations 
from practicing school leaders and personal interviews) for making admission decisions 
(Browne-Ferrigno & Shoho, 2002), such information has not been related to subsequent 
performance in the program. 

Recent suggestions urge faculty to consider additional information in making admission 
decisions, such as candidates’ ability to reflect and their commitment to social justice (Grogan & 
Andrews, 2002). Nevertheless, current practices provide little information about what students 
know and are able to do when they begin the program — and how such knowledge and skills can 
enhance learning processes. Without such data, it is virtually impossible to determine with any 
confidence just how (or whether) a program influences students’ knowledge, skills, or attitudes. 
Thus, baseline data need to be obtained and compared to similar assessments throughout the 
program, at its conclusion, and as the graduates begin their first — and later — administrative 
positions. One way to obtain pre-program data is to use performance assessments, such as those 
used in the NASSP Assessment Center, that are related to what is exemplary about on-the-job 
performance. Besides obtaining written information, some programs require candidates to 
complete in-basket exercises, make presentations, engage in small-group activities, and 
participate in interviews, which are evaluated by faculty and practitioner teams (Milstein et al.. 
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1993). In this way, candidates’ performances can be used not only for admission decisions but 
also for comparisons with similar assessments conducted later on. 

The standards-based approach advocated by Glasman et al. (2002), for example, could be 
used to determine types of information to obtain about knowledge, skills, and dispositions. In 
addition, educational leadership programs that align their programs with national standards, such 
as those advocated by ISLLC or NCATE, can use these standards to guide data collection at all 
points along the way. Besides becoming benchmarks for admission and student performance, 
these data can serve as formative assessments for ongoing program revisions. Regardless of the 
data collected, some data need to parallel those collected prior to the program, thus ensuring 
comparability for ascertaining growth in knowledge and skills and their potential transfer to 
practice. 

Finally, data can be gathered after graduates have completed the program. On the one 
hand, information can be obtained as they begin new school leadership positions. For instance, 
studies of beginning principals provide valuable data about how preparation programs have 
affected their attitudes and performance (Barnett, 2001; Berg, 2001; Peel et al., 1998), as well as 
the progressing stages of development experienced by school leaders (Hart, 1993; Weindling, 
2000). In addition, research designs can obtain information from co-workers about graduates’ 
leadership skills (Leithwood et al., 1995). On the other hand, follow-up investigations with 
graduates who have not taken formal leadership positions also can reveal how programs have 

influenced their workplace attitudes and behaviors. 

/ 

Comparability of Data 

One of the advantages of gathering comparable data at the regional or national level 
would be to determine trends and patterns in leadership preparation. However, when collecting, 
maintaining, and publicizing these assessments, several precautions should be taken. First, 
comparisons must be sensitive to local variations and contexts. Individual programs need to 
determine what data to share publicly — perhaps agreeing to and being provided standardized 
guidelines for data presentation — without using a “one size fits all” approach. In this way, 
professors and programs could build ownership in data collection and distribution processes. 
Second, the same type of information must be available across programs. This requires 
agreements about the domains to be assessed, which might include the performance standards of 
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professional associations or the outcomes-based standards identified by Glasman et al. (2002). 
The EEELP initiative is particularly alert to this issue, urging nevertheless that core-learning 
competencies and instructional standards must be identified, measured, and reported (Orr, 2002). 
Finally, when information is compared and contrasted across programs, making relative value 
judgments about whether programs are excelling or floundering should be avoided. If this data- 
collection endeavor is perceived to be a norm-referenced summative assessment, then faculty 
will be very reluctant to participate. If, however, information is available about the types of data 
being collected for different learning domains or core competencies and how students in 
preparation programs perform in these areas, then the profession can learn how programs affect 
students’ skills and knowledge without chastising or punishing individual programs. 

Data Collection Agendas 

Several ongoing projects point to the importance of collecting data about the performance 
of students in educational leadership preparation programs. As mentioned earlier, the goals and 
expectations of the EEELP and the NCAELP underscore the importance of gathering and sharing 
inter-institutional data. Although these calls to arms are essential in pointing the profession in 
the right direction, such initiatives clearly need an organizational structure and funding base to 
implement their goals (Orr, 2002). Professors acting alone will have considerable difficulty 
mustering the human and financial resources needed to conduct such a massive effort. What 
needs to occur — the driving force behind the formation of the NCAELP — is a collaborative 
approach because “no single organization, group or individual can create the kind of changes for 
leadership preparation that our nation’s children need and deserve” (Young et al., 2002, p. 140). 

To advance current collaborative plans, it is worth considering merging the efforts of the 
EEELP and NCAELP (and, perhaps, UCEA’s center). Not only would professors feel their 
involvement in reform is appreciated and respected, the work already completed by EEELP could 
provide momentum and direction for NCAELP. Besides creating common goals and operational 
structures, this consortium would need to search for external funding to carry out the type of data 
management processes already outlined. Without these resources, efforts may fall on individuals 
with other demanding professional commitments and lack focused attention. 

Conducting meetings and commissioning articles is a very useful first step, yet the serious 
work of designing and implementing data collection efforts of the scope proposed requires long- 
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term commitment. Although hiring data collection and management specialists for short time 
periods would be valuable, having a cadre of full-time professionals devoted to this task is 
essential. . Without the necessary human and financial capital, a coordinated data collection effort 
is not likely to be realized, and the opportunity to influence educational leadership preparation 
and promote national professionalization will be lost. 

Developing Long-term, Nationwide Collaborative 
Research Partnerships and Strategies 

A nationwide longitudinal study about the effectiveness of cohorts in university-based 
educational leadership programs is needed. Undertaking such an expansive research effort, 
however, is not for the faint of heart. The challenges in developing a long-term collaborative — 
and sustainable — research design that provides meaningful and comparative findings across 
university and state boundaries need to be identified and addressed. In addition, an investigation 
of this proposed magnitude may well influence the field both positively and negatively. Thus, 
the research designers should be clear about what needs national attention and what 
consequences might follow a focus on those needs. 

Broad-based partnerships, for example, are necessary to ensure that a focus on cohorts 
connects to desired reform outcomes. Inclusive collaboration in design and implementation 
involves a critical mass of leadership educators and preparation programs as well as 
representatives of professional organizations, accreditation agencies, and policy-study groups. 
The significant body of extant literature, both research based and critical of current preparation 
programs, provides rationale and guidance for conducting such large-scale inquiry. Nonetheless, 
important design issues must be clarified, time lines established, and actions — not continued 
rhetoric — initiated . 

Developing an Agenda for Reform: Comparability and Sustainability 

A first decision to be made in designing a nationwide collaborative study is to determine 
what types of cohorts within administrator preparation programs will be studied. While the 
current spotlight is on the effectiveness of programs that prepare K-12 school leaders, all 
graduates from educational administration programs ultimately influence the effectiveness of 
administrative practice. Thus, will outcome studies purposefully limit their focus only to cohorts 
leading to credentialing of K-12 school administrators, or will they include doctoral cohorts that 
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prepare district and college administrators and new entrants into the educational administration 
professoriate? Further, many preparation programs are delivered through differing formats to 
meet specific constituent needs or environmental influences. Some cohorts meet face-to-face 
regularly at local sites. Other cohort programs are delivered predominately through distance- 
learning technologies with asynchronous online participant interaction and limited, if any, face- 
to-face connections among students and instructors. Thus, will these studies be limited to face- 
to-face cohorts, or will they also include virtual cohorts? 

A second consideration in planning nationwide studies of cohorts is establishing the 
criteria for comparable units, namely defining what a cohort is. Most research has been limited 
to students that “take all or a significant portion of their course work with an intact group rather 
than randomly enrolling in courses at their own pace” (Barnett et al., 2000, p. 1). Using this 
definition of a cohort may limit findings about the influence of preparation programs on eventual 
administrator practice. What about loosely coupled “cohorts” of students that begin programs 
together and follow a prescribed sequence of courses that also includes students who do not 
follow a fast-track completion schedule or who take only selected courses as part of other 
administrative certification programs? Do quasi-cohorts (i.e., groups of learners formed only for 
intensive orientation or formed informally by course scheduling) need to be included because 
loosely coupled “cohorts” may provide peer support and group learning that influences later 
professional practice? Might the sequencing or types of courses or other learning activities need 
to be considered to see if they have greater effects than those attributable to how students 
progress from start to completion of a program? 

Additionally, research reports about cohorts have most often emerged from studies 
focused on specific groups of 15 to 25 learners that spanned inquiry intervals of 3 to 18 months 
or used data collected at the close of programs, thus providing only snapshots of the cohorts. 
Many preparation programs leading to K-12 administrator certification or licensure extend over 
two or more years, yet little is known about the benefits or drawbacks of extended cohort 
membership (Maher, 2001). Thus, do issues about cohort membership size and their time 
duration need to be clarified in order to compare findings across sites? Comparability of findings 
may also be influenced by faculty perceptions about the value of using cohorts in leadership 
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preparation programs (Barnett et al., 2000), perhaps introducing researcher bias. What strategies 
need to be used to counteract potential limitations due to bias for or against cohorts? 

Administrative licensure of K-12 school leaders is controlled by state agencies that have 
adopted professional standards for the preparation, evaluation, and practice of educational 
administrators. Thus, preparation programs, approved by state agencies and accredited by 
NCATE, are grounded upon administrator standards (Browne-Ferrigno & Muth, 2002; Muth, 
2002a) that define preparation and practice outcomes. Forty states have adopted the ISLLC 
Standards or correlated their standards to align with the ISLLC recommendations, while ten 
states have either adopted their own administrator standards or eliminated licensure completely. 
Hence, another design issue related to comparability of findings involves determining what 
students need to know and be able to do as effective school principals. A nagging problem arises 
again: How do we define an effective principal? Do ISLLC or NCATE standards directly link to 
leadership efficacy? If so, in what ways? 

Another consideration in the research design concerns sustainability. Undertaking 
longitudinal studies that span several years and involve multiple inquiry sites require researcher 
commitment and institutional support. Thus, will study sites be limited only to cohorts at 
research universities where faculty time and effort is oriented toward inquiry, or will such studies 
include institutions in which faculty focus primarily on teaching and service? And, does 
institutional type or orientation have an affect on program outcomes and later practice? 
Additionally, collecting program participant data at intervals recommended earlier in this paper 
(before, during, at the end, and after the program) requires institutional approval for human 
subjects study and deliberate attention to meeting time-sensitive benchmarks. Careful and 
collaborative preplanning and research oversight is required to ensure that findings yield valid 
and comparable data. 

Another challenge to project sustainability relates to the researchers involved. Because 
this proposed investigation is intentionally long term, how will local studies be designed to 
combat researcher relocation or retirement? Sustainability of research also requires tracing 
graduate career progress following program completion. Thus, databases will be needed that are 
continually updated to track contact information. Maintaining graduate databases within 
administrator preparation programs would be ideal, but the reality is that programs seldom have 
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the resources or even the interest in database management. Labor costs are immense, and most 
programs nationally are quite small and lack resources. Potential support for tracking graduate 
whereabouts may be available from state licensing agencies, university alumni organizations, or 
even school-district offices. However, release of personal information may be restricted by state 
statute. Thus, study sites will need to develop mechanisms for tracking program graduates over 
their careers in educational leadership. 

Determining the Agenda: First Steps, Next Steps 

Reviews of the need to reform university-based administrator preparation programs 
became extensive in the mid-1980s and after. Since then, more than a decade of responses have 
ensued (Forsyth & Murphy, 1999; Griffiths, Stout, & Forsyth, 1988; Milstein & Associates, 

1993; Milstein & Krueger, 1997; Murphy, 1992, 2002; Usdan, 2002). Researchers from both 
UCEA and non-UCEA institutions have studied cohorts and shared their findings through 
presentations at professional conferences, articles in journals, and chapters in books. Their 
conclusions repeatedly cite a need for more research. Thus, the time is now (McCarthy, 2001) 
for teams of dedicated researchers to develop proposals for long-term, nationwide, collaborative 
research designs that allow flexibility and encourages the participation of most preparation 
programs. 

The Future: Both Positive and Negative 

Launching a nationwide collaborative research agenda about cohorts presents several 
major stumbling blocks. The first, and perhaps the most critical, is maintaining an invitational 
stance that encourages many to join the inquiry. Investigations need to include those professors 
who perceive both benefits and disadvantages to using cohorts in administrator preparation 
programs. Because the purpose of this nationwide study is to gather empirical evidence about the 
influence of cohorts on professional practice of educational practitioners, project designers need 
to seek involvement and commitment from a broad group of researchers and supporters. Long- 
term, nationwide studies of cohorts need to acknowledge existing realities: Leadership 
preparation programs, even within the same state, vary considerably due to university or college 
purposes and service orientations, political and locational conditions and constraints, and 
departmental composition and preferences. 
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Nationwide involvement by a wide field of investigators can create a second stumbling 
block: engaging a committed team of experienced and novice researchers to sustain the project. 
Conducting longitudinal studies requires continuing researcher and participant attention, which 
can become difficult to maintain over time. Thus, the research design must include strategies to 
ensure continuing participation at study sites. Another stumbling block linked to sustainability is 
financial support. The project needs to be coordinated by a group that has the credibility to 
attract and maintain the resources necessary to dedicate time and personnel to the project’s 
maintenance delivery of promised products and outcomes. Unless faculty can gain releases from 
other responsibilities, the demands of coordinating a national effort and connecting to multiple 
sites will become too onerous to be handled by a few far-flung professors. Thus, a Center for the 
Study of Program Effectiveness needs to be created — perhaps as part of the EEELP or NCAELP 
efforts — and supported by external monies sufficient to develop long-term, coherent baseline 
data collection processes. 

A third potential stumbling block for this project involves overcoming the dualities 
common to many universities and colleges. Most professors must conform to restrictions that 
focus on stability of expectations, even while engaging in innovative applications of and research 
on instructional models that meet demands of the field, of innovators in teaching and learning, 
and of accrediting agencies. Another issue associated with long-term studies is the cessation of 
cohort models because of changed institutional missions or support resources, decreases in 
students, or the exit of key faculty. 

Even given these negatives, developing a research project that clearly outlines goals, 
objectives, and tasks to be accomplished can consolidate many efforts. First, university programs 
or individual researchers can use the project to mold research designs to unique local contexts, 
and individual study sites can identify to which parts of the broader research agenda they can 
contribute. Second, and critically important to the success of this endeavor, a broad-based 
research project is more likely to attract significant external funding. Prospects for receiving 
major financial support may be greater if cohort-specific research becomes a fifth component of 
the EEELP project, further diminishing the appearance of disconnected research priorities. 
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